1. Field of the Invention
This invention relates to providing light for forming images and, more particularly, to providing light for forming images for a passive viewer.
2. Description of the Prior Art
Still photography, motion pictures and television were influenced by the way artists represented physical reality in paintings, as if through a window. A highly detailed perspective image is provided, typically within a rectangular frame. All provide highly detailed images which induce the viewer to cooperate with the cameraman""s xe2x80x9cvisionxe2x80x9d by assuming the artificial perspective of the representation. The viewer is enabled to deliberately suspend disbelief that the images themselves are not a real object space. The degree to which the viewer is thus enabled is influenced not only by the image resolution but by the field of view. It is usually thought desirable to increase both. For example, very high resolution commercial television standards have been formulated for increasing image quality. Such approaches typically increase the number of horizontal lines scanned to a number significantly greater than present standards. Larger format movie film such as 70 mm has been used to increase detail. Also, panoramic movies, e.g., xe2x80x9cCineramaxe2x80x9d increased the field of view to increase realism. Various stereoscopic television approaches have also been conceived or developed to increase realism.
All of these traditional media take a rather objective view of the physical world. The image is framed by a window through which the viewer can gaze in any direction xe2x80x9cintoxe2x80x9d a representation of an object space. Events are presented in both movies and television in a series of different action scenes in a story line which the viewer can observe from a stable and seemingly quasi-omniscient point of view. The viewer is led to take what appears to be a view of the world as it really is. Yet the choice of image and its perspective is picked by the creator of the image and the viewer actually assumes a passive role.
A sensorama simulator was disclosed by Heilig in U.S. Pat. No. 3,050,870. The senses of an individual were stimulated to simulate an actual experience realistically with images, a breeze, odors, binaural sound and even motion. Heilig also disclosed a stereoscopic television in U.S. Pat. No. 2,955,156. This also was passive. xe2x80x9cVirtual reality,xe2x80x9d in an electronic image context, goes even further in the direction of increased realism but enables the viewer to take a more active role in selecting the image and the perspective. It means allowing a viewer""s natural gestures, i.e., head and body movements, by means of a computer, to control the images surroundings, as if the viewer were seeing and moving about in a real environment of seeing, hearing and touching. Due to the myriad of possible actions of the viewer, a corresponding multiplicity of virtual activities needs to be available for viewer choice. This would represent the ultimate in artificial experience.
A user of a xe2x80x9cvirtual realityxe2x80x9d device will typically don a head-mounted display which provides images of a virtual space that are matched to the sensed position and orientation of the head of the user as the user moves his head in space and time (e.g., the x, y, z position of the head and/or the roll, pitch, yaw attitude of the head). For example, a Fakespace BOOM3C is a Binocular Omni-Orientation Monitor that provides visual displays and tracking integrated with a counterbalanced articulated arm for full six-degree of freedom motion (x, y, z, roll, pitch, yaw) and provided by Fakespace, Inc., 241 Polaris Ave., Mountain View Calif. 94043. Another example would be a wireless magnetic motion capture system such as the STAR*TRAK of Polhemus Incorporated of 1 Hercules Drive PO Box 560 Colchester Vt. 05446. It provides six-degree-of-freedom (position and orientation) data from up to 32 sensors capturing data at up to 120 Hz.
The images for such devices are created by a computer program with the assistance of pre-stored image information that is retrieved according to the user""s head movements and presented to the user""s eyes. The user""s head may be coupled to the display. The aim is to present panoramic images covering a wide field of view in order to immerse the user in an artificial reality with which he can interact, as if real. The degree of artificiality need not be total and can instead constitute an xe2x80x9caugmented realityxe2x80x9d with some artificial objects or symbols superimposed or interposed within the real world as viewed with a see-through, head-mounted or head-coupled display.
These advances take advantage of converging technological developments in telecommunications including broadband services, projection optics for head mounted and head-coupled displays (including virtual retinal displays), the ever-increasing computational power of image processing computers, specialized sensors such as gloves designed to sense hand and finger movements, exoskeletons, and the like. They can be expected to lead to exciting interactive games and other new forms of interactive experiences within virtual worlds.
This new paradigm represents a very great improvement over the present imaging technology. It joins immersion to interactivity to increase the level of experience. It is now being applied to gaming applications and others such as virtual museums, architectural and interior design mockups, facility tours, xe2x80x9caircraftxe2x80x9d rides and the like.
The new paradigm would likewise seem to hold the potential for an improvement over the old ways of traditional entertainment such as drama, comedy, documentaries, and the like. By joining immersion and interactivity, the user would be enabled to enter a completely new realm of artificial experience. The user would be given a very high degree of freedom, under his own volition, to navigate in the virtual world and to participate in completely new forms of such entertainment, where the user""s own actions influence the sequence of images and audio provided.
Traditional entertainment applications, on the other hand, such as drama, comedy, documentaries, and the like, have not yet been explored by these new technologies. This could be because the traditional applications have usually been presented for passive enjoyment by the viewer. Even though immersion would provide a better experience, interactivity would be contrary to these known traditional entertainment applications, such as storytelling, where people like to relax and be passively led through stories and participate vicariously. Another obstacle would seem to be that the level of complexity of the possible alternative scenarios, depending on the user""s actions, would need to be higher in the traditional arts than for the more predictable and mechanistic art of gaming.
For all these various kinds of virtual reality applications, the creation of many possible scenarios for viewer selection creates a massive demand for electronic image storage space and there is also the problem of a disconcerting time lag between the viewer""s action and the response of the imaging system. These problems make this emerging technology hard to achieve using presently available hardware. The software task is equally daunting.
An object of the present invention is to provide a new method and means of providing light for forming images for a viewer.
According to the present invention, a method of providing light from a light source at an orientation of the light source to an eye in a head of a viewer for formation of images in the eye, comprises the steps of:
providing the light from the light source for the formation of images with a changing point of view; and
changing the orientation of the light source in correspondence with the changing point of view for guiding the head of the viewer in a correspondingly changing orientation for viewing the images with the eye in the head of the viewer at the changing orientation of the light source and from the changing point of view.
The present invention may be carried out by apparatus, comprising:
a light source, responsive to a light control signal, for providing light for viewing images by an eye in a head of a passive viewer; and
a light source actuator, responsive to a head guide control signal, for causing the light source to execute attitudinal movements for emulation by the head of the passive viewer.
The actuator may be a robot configuration selected from the group consisting of Cartesian, cylindrical, spherical, and articulated robot configurations.
The images may but need not be created by means of one or more cameras associated with a cameraman, for example, on the head of a cameraman. These are provided, according to the invention, for passive perception by a viewer whose head movements are guided by a motion-controlled head guide that is actuated in such a way as to emulate head movements of the cameraman in synchronism with the images actively sensed by the cameraman. The xe2x80x9ccameraman,xe2x80x9d if there is such, can but need not have one or more cameras mounted on his head and the direction of his head with respect to a selected reference frame is monitored; head monitoring signals are stored in association with individual images picked up by the head-mounted camera or cameras. Such images are provided xe2x80x9clivexe2x80x9d or are played back to the passive viewer by way of a display fixed on or in the head guide, e.g., by way of a headup display fixed to the head guide. The motion of the head guide is controlled with respect to the individual images by retrieving the previously stored head monitoring signals in synchronization therewith. The head of the passive viewer is urged by the controlled movements of the head guide to execute head movements emulative of the monitored motions of the cameraman at the time of image acquisition.
Simulated active percepts, according to the present invention, permit a viewer to experience percepts passively, as if inside the head of another person. This xe2x80x9cother personxe2x80x9d is the xe2x80x9conexe2x80x9d controlling the acquisition of the percepts experienced by the passive viewer. Even though the images presented to the passive viewer may be panning about and changing perspective at the whim of the xe2x80x9cother person,xe2x80x9d e.g., the cameraman, the passive viewer has those images presented to his eyes while his head is also urged to move in the same direction as that of the cameraman""s head so that it is directionally coordinated with the images viewed by the cameraman, as if he were viewing them himself, through his own eyes.
It should be realized that cameras are not needed and the images can be created by means of a computer workstation or even by known animation techniques coupled with computers and/or cinematography. In that case, the head movements can be preplanned rather than sensed.
There can be a large number of passive viewers with their own motion-controlled head guides. These can be embodied in second-hand (passive) experience simulators, e.g., in the form of self-contained booths each with a multi-degree of freedom head guide for connection within. The viewer""s head guide may be actuated in any number of degrees of freedom, as a matter of design choice, to exert some minimum degree of mechanical head guidance control with just a few actuators or can provide a full complement of actuators, e.g., providing control in six or even more axes. A booth can be for home or arcade use, for example. Such a viewer enters the booth, sits down and mechanically couples his head to the head guide. E.g., the display may be a panoramic display fixed in the wall of the booth or may be a helmet mounted display, as known in the art. The invention need not be embodied in a booth. It can be desk mounted or mounted in any convenient way.
The images provided to the passive viewer""s eyes can be varied in their apparent distances, e.g., by changing the focus of the optics in an eyepiece of the light source. In this way, the accommodation of the eyes of the viewer can be urged to follow the changes in focus at differing depths within the image space.
The invention may be made even more like a re-experience of experiences of another, according to another aspect of the present invention, by effectively controlling eye movements of the passive viewer in such a way as to emulative of eye movements of the other, e.g., the cameraman. This can be done in a nonintrusive way by presenting nonuniform images emulative of the human fovea, e.g., with nonuniform resolution, nonuniform dynamic range, a small colored area in an otherwise wide-field black and white image, nonuniform image informational content, nonuniform image concentration, nonuniform brightness, or some other equivalent nonuniform images to the passive viewer, made so as to draw the viewer""s attention to an accentuated area, wherein such area moves about between successive images presented within the field of view of the viewer. In this way, not only the head of the passive viewer has its motions guided but the eye movements are guided as well. So the passive viewer can have his head guided to be directed in one direction while the attention of his eyes is drawn or guided in another direction. In this way, the passive viewer feels even more like he is undergoing experiences of another, e.g., the cameraman. Such images can be created by monitoring one or both eyes of the cameraman and causing the image information gathered by the cameras to be encoded in a nonuniform way such as by having finer scanning in a small area dictated by where the cameraman happens to be looking at a given moment with the rest of the field scanned coarsely.
Furthermore, when coupled with the previously described changing focus for changing the apparent distances of the images, the foveal viewing aspect of the invention can be used to xe2x80x9ccontrolxe2x80x9d the exact point of fixation to which the passive viewer""s visual attention is directed, thereby establishing a sequence of fixations at various points at various depths in the image space with correspondingly differing accommodation and convergence of the viewer""s eyes.
Such simulated active percepts may be presented xe2x80x9clivexe2x80x9d or may be stored and retrieved from storage and later presented for passive perception. A booth can, for example, be provided with a video cassette recorder to playback the image and head guide control information. The stored imagery could even be downloaded or provided xe2x80x9clivexe2x80x9d from the Internet.
In the case of stored simulated active percepts, according to the teachings hereof, since there is only one set of images to store, the massive memory demand problem of the prior art of xe2x80x9cvirtual realityxe2x80x9d is solved. Similarly, for the xe2x80x9clivexe2x80x9d case, since the simulated active percept is provided at the same time as it is created, there is no storage requirement at all, i.e., beyond temporary, xe2x80x9con-the-flyxe2x80x9d storage needs.
Moreover, by providing simulated active percepts for passive perception, there is no longer any time lag or latency problem as is presently the case for known virtual reality applications. Since the simulated active percepts induce the passive viewer to emulate those physical actions which caused or would have caused the simulated active percepts, the hardware need not be faster or as fast as the viewer. In fact, it may be much slower. Although the viewer is relegated to a passive role, the novelty and richness of the xe2x80x9cvirtual reality,xe2x80x9d immersive experience more than compensates in opening a whole new world of opportunity for representing reality.
These and other objects, features and advantages of the present invention will become more apparent in light of a detailed description of a best mode embodiment thereof which follows, as illustrated in the accompanying drawing.